1. Field of the Invention
The present invention relates to an information input/output device, an information input/output method and a computer program performing input/output of information in response to user operation, and particularly relates to an information input/output device, an information input/output method and a computer program performing operation with respect to virtual space through user operation in real space as well as performing output of information so as to feed back a force sense from virtual space.
Further more particularly, the invention relates to an information input/output device, an information input/output method and a computer program using an operation area Sd in real space and performing operation in an operation area Sv in virtual space which corresponds to the operation area Sd, particularly, relates to an information input/output device, an information input/output method and a computer program performing output of information so as to provide a user with a tactile sense from all over the virtual space by expanding an accessible range in virtual space by moving the operation area Sv in virtual space which corresponds to the operation area Sd in real space in accordance with user's intent.
2. Description of the Related Art
In a technical field such as virtual reality and telereality, a force/tactile display, namely, a “haptic device” for feeding back a force sense or a tactile sense to an operator in addition to visual information and hearing information is indispensable.
Along with improvement of computer speed and progress of a simulation technology in recent years, virtual environment in which plural objects coexist and physical interaction such as collision or contact occurs between them can be simulated in real time. When collision between objects or a contact force at that time can be calculated exquisitely as well as in real time in consideration of dynamics as described above, senses when the user touches or grips an object in the virtual environment through the haptic device can be fed back to the user with reality by actually generating the calculated force by a motor.
Applications of the haptic device are wide. In learning of special skills such as medical care or the like, remote operations in virtual environment such as microcosms or oceans and in special or dangerous environment such as a nuclear reactor, a three-dimensional force/tactile senses with respect to objects in environments which are difficult to be actually touched can be fed back. According to scale or accuracy of virtual space which can be processed in real time, increase of demand of the force/tactile display is expected in the future.
For example, a pen-type haptic device using a serial link as shown in FIG. 16 is common. The user grips a tip portion of a pen shape, in which the 3-axis force to 6-axis force moment is fed back.
In addition, a haptic device feeding back the 3-axis force to 6-axis force moment by using a parallel-link structure is proposed as a structure for solving lack of stiffness of the serial link as shown in FIG. 17.
In any of the above haptic devices using the link mechanisms, force/tactile feedback is performed back with respect to only one point of a gripping position, in other words, the force/tactile feedback is not performed to respective fingers of an operator independently. Accordingly, there is a problem that it is difficult to feed back a sense of gripping a virtual object by fingers.
On the other hand, when using a multi-finger type haptic device, the operator can obtain a sense as if he/she grips a work object. This kind of haptic device is applied to a master hand which remotely operates a slave hand such as a robot hand, thereby having a function of transmitting motion of the master hand (operation means) to the slave hand (work means) as well as feeding back a force sense of the slave hand side to the mater hand.
In order to feed back the force to plural fingers, a haptic device in which fingers and remote motors are connected by wires and the force is applied to respective fingers through the wires as shown in FIG. 18 is proposed. However, in such device structure, it is necessary to control so that the wires are not slackened at any time, which may complicate the control. In addition, the interference between wires is liable to occur, particularly, movable range concerning rotation is not so wide.
A haptic device in which a hand-type force feedback mechanism is added to an opposed manipulator as shown in FIG. 19 is also proposed (for example, refer to Attributed to Haruhisa Kawasaki, Takumi Hori and Tetsuya Mouri “Opposed multi-finger haptic interface” (Journal of the Robotics Society of Japan Vol. 23, No. 4, pp 449-456, 2005) (Non-Patent Document 5)). As a method similar to this, a method of adding the hand-type force feedback mechanism to tips of an exoskeleton is also proposed. In addition, a method of feeding back 3-axis force to fingers by installing fingerstalls at a tip of the serial link is also proposed (for example, refer to JP-A-2004-299999 (Patent Document 2)).
The virtual space in which simulation is performed extends endlessly, whereas, since an operation area of the haptic device has constraints in movable range of the device, the operation area is limited, and the user accesses only to part of virtual space corresponding to the limited operation area. In FIG. 20, an operation area Sd of the serial-link structure haptic device in real space and an operation area Sv corresponding to the operation area Sd in virtual space. The operation area Sd has constraints of movable range of the haptic device. Therefore, the user can touch an object in the operation area Sv in virtual space corresponding to the operation area Sd of the haptic device, however, it is difficult that the user touches an object outside the operation area Sv. For example, in the drawing, the user can receive a tactile sense (force feedback) with respect to an object A in the operation area Sv, however, it is difficult that the user obtains the tactile sense with respect to an object B which is outside the operation area Sv.
In order to touch the object B which is outside the operation area Sv, that is, in order to obtain the tactile sense by the user from the object B through the haptic device, it is necessary to move (parallel movement) the operation area Sv in virtual space corresponding to the operation area Sd of the haptic device in real space to a position including the object B.
Naturally, the range of virtual space which can be accessed by the user is expanded when the movable range of the device like the serial-link mechanism is expanded, however, which makes the device large in size and incurs cost increase.
In a field of GUI (Graphical User Interface), it is possible to move the operation area and allow a mouse cursor to access all over a GUI screen by performing an operation of lifting a mouse every time the mouse reaches edges of a mouse pad and arranging the mouse on the mouse pad again, namely, “Indexing” is performed repeatedly. However, the operation of “lifting to rearrangement” corresponds to an operation of stopping processing of movement of the mouse cursor on the screen and performing the processing again, therefore, the operation is not applied to the haptic device on the basis of continuous input.
Additionally, a method of giving an offset instruction of the operation area Sv in virtual space by using an input interface such as a switch together. However, the operation will be complicated when the input degree of freedom of the haptic device is increased, which prevents intuitive operation. For example, in the case of operation of the opposed manipulator shown in FIG. 19, the input degree of freedom opposed according to fingers can be sensed intuitively as a continuous operation of respective fingers, however, the input interface operation which gives the offset instruction of the operation area Sv is not continuous, therefore, it is difficult that the operation is intuitive.
According to the above reason, it is necessary that the operation area Sv in virtual space is moved only by the original input operation of the haptic device performing operation in virtual space. Alternately, when the offset of the operation area Sv in virtual space is performed by detecting intent of the user based on the input operation with respect to the haptic device, the operation becomes intuitive and natural, which largely improves user-friendliness of the haptic device.
The relationship as shown by the following formula is established between a position “rd” of a tip of the haptic device and a position “rv” of a pointer in virtual space as shown in FIG. 20.rv=rv0+Krd  (1)
In the above formula (1), “rv0” represents a position of the origin of the haptic device in virtual space. “K” is a scale factor, which provides a scale between actual displacement of the haptic device and displacement of the pointer in virtual space.
As can be seen from the above formula (1), when the scale factor “K” becomes large, the pointer in virtual space can be largely moved by small displacement of the haptic device, as a result, it is possible to expand the operation area Sv in virtual space which can be operated by the same operation area Sd in real space of the haptic device. That is to say, if the haptic device is small, the device can touch large virtual space. However, positional resolution is reduced at the same time when the scale factor “K” becomes large, therefore, there occurs a problem that the tactile sense is reduced. In addition, to change the scale factor “K” is different from to give the offset to the operation area Sv, therefore, even when virtual space touched by the device is expanded, it is still difficult to touch objects outside the fixed operation area Sv in the virtual area.
Furthermore, a method in which a scale factor between minute displacement “δrd” of the haptic device and minute displacement “δrv” of the pointer in virtual space is changed according to speed “vd” of the haptic device, which is called as “ballistics” can be cited. As a rule, setting is performed so that the higher the speed of the haptic device is, the larger the scale factor “K” becomes. Accordingly, when precise interaction in which speed of the haptics device is low is performed, lack of resolution can be avoided.δrv=K({dot over (r)}d)δrd  (2)
However, in the “ballistics”, the position “rv0” which is the position of the origin of the haptic device in virtual space is not managed, therefore, the operation area Sd of the haptic device is gradually shifted from the operation area Sv in virtual space. Accordingly, there is a disadvantage that a situation that operation has to be performed in the vicinity of boundaries in the operation area Sd of the haptic device often occurs.
There is also a method of using the haptic device as a speed operation device, which is called as a “rate control”. In the method, speed of the pointer in virtual space according to the tip position “rd” of the haptic device is provided, which is represented as the following formula (3).{dot over (r)}v=Krd  (3)
According to the “rate control”, it is possible to access all over the space. However, motion of the haptic device is given to the pointer by performing integration, therefore, it will be a first-order lag system and there are problems that it is difficult to obtain a tactile sense of a rigid body and that it is difficult to perform rapid motion instruction.
Moreover, a method of changing a scale factor of actual displacement of the haptic device with respect to displacement of the pointer in virtual space, and a method of changing speed of the operation area Sv in proportion to speed and a position of the haptic device based on knowledge of psychophysics in human beings as a hybrid method which combines the ballistics with the rate control as shown in the following the formula (4) (For example, refer to Attributed to F. Conti and O. Khatib “Spanning Large Workspaces Using Small Haptic Devices” (IEEE World Haptics 2005) (Non-Patent Document 7)).{dot over (r)}v=K|{dot over (r)}d|rd  (4)
In the hybrid method, there is a problem that the operation area is not moved when speed of the haptic device becomes “0”, even if the user desires to reach farther space. In addition, since the operation area Sv moves also during contact interaction such as grazing a floor, the haptic device may give a sense of incongruity such as “slipping” to the user.